FIGHT'S 


Published  by  the  Warfighter  Readiness  Research  Division 

711th  Human  Performance  Wing  /  RHA  of  the  Air  Force  Research  Laboratory 

Human  Effectiveness  Directorate 


Volume  8,  Issue  III 
FALL  2009 


Tactical  Digital  Holograms  Favored  Over  Conventional  2D  imagery 


Tactical  digital  holograms  (TDH)  are 
large-scale  3D  static  imagery  presenta¬ 
tions  that  resemble  large  sheets  of  x-ray 
film  and  allow  3D  visualization  of  any 
object,  geographic  area,  or  other  complex¬ 
ity  that  can  be  digitally  modeled.  TDHs 
are  reflective  and  must  be  illuminated  for 
viewing,  but  the  illumination  may  be  from 
any  point  light  source;  no  special  pro¬ 
jectors  or  light  sources  (e.g.,  lasers)  are 
required.  A  typical  TDH  is  comprised  of 
over  500,000  individual  holographic  ele¬ 
ments  (hogels),  with  each 
hogel  containing  up  to  a 
million  different  perspec¬ 
tive  views.  TDHs  thus  pro¬ 
vide  full  image  parallax, 
e.g.,  simultaneously  cor¬ 
rect  viewing  perspectives 
for  all  individuals  viewing 
them. 

TDHs  have  been  opera¬ 
tionally  employed  by  the 
Army  as  mission  planning 
tools  for  several  years, 
and  demand  has  steadily 
grown,  but  initial  informa¬ 
tion  on  their  utility  was 
anecdotal  in  nature  with 
little  or  no  quantitative 
data  available  to  evaluate 
their  value.  In  2007-2008, 

711  HPW/RHA  (Mesa 
Research  Site)  conducted 
the  first  research  to  begin 
quantifying  the  utility  of  TDHs.  The  Mesa 
Research  Site  centered  on  close  air  sup¬ 
port  mission  planning  and  execution,  with 
specific  focus  on  the  joint  terminal  attack 
control  (JTAC)  mission,  and  was  con¬ 
ducted  at  the  9th  Air  Support  Operations 
Squadron  (the  JTAC  “school  house”)  at 
Fort  Hood,  TX.  Due  to  the  Army’s  previ¬ 
ous  use  of  TDHs,  711  HPW/RHA  invited 
the  Army  Research,  Development  and 
Engineering  Command’s  Simulation  and 
Training  Technology  Center  (RDECOM- 
STTC),  located  in  Orlando,  FL,  to  collab¬ 
orate.  The  study  utilized  written  realistic 
mission  scenarios,  individual  interviews, 
and  structured  questionnaires  in  which 


personnel  subjectively  rated  the  utility  of 
TDHs  and  conventional  2D  maps  of  the 
same  area,  at  the  same  scale  and  resolu¬ 
tion,  for  13  specific  tasks  using  a  scale 
of  1  (poorest)  to  10  (best).  The  results 
clearly  indicated  that  TDHs  can  be  an 
effective  tool  for  improving  JTAC  mis¬ 
sion  planning  and  execution  tasks.  The 
most  frequently  cited  benefits  included 
determination  of  relative  heights  of  build¬ 
ings  and  other  objects,  lines  of  sight  and 
fire,  positioning  of  teams  and  snipers,  and 


Representative  tactical  digital  hologram  of  Urban  Area  with  3D  Height 

JTAC  over- watch  locations.  Averaged 
across  all  tasks,  users  rated  the  utility  of 
conventional  2D  maps  at  5.2,  whereas  the 
utility  rating  for  TDHs  was  8.2. 

Building  on  711  HPW/RHA  research, 
RDECOM-STTC  subsequently  conducted 
research  to  collect  data  quantifying  the 
value  of  TDH  imagery  in  the  performance 
of  certain  Army- specific  mission  scenar¬ 
ios.  In  May  2009,  RDECOM-STTC  con¬ 
ducted  research  on  TDH  effectiveness  at 
Camp  B landing,  FL,  in  which  they  invited 
711  HPW/RHA  to  collaborate.  The  Mesa 
Research  Site  contributed  significant  por¬ 
tions  of  the  experimental  protocol.  Using 


visual  stimuli  based  on  identical  sample 
data  sets,  subjective  utility  rating  data 
were  collected.  A  comparative  analysis  of 
time  and  accuracy  assessed  2D  conven¬ 
tional  tools  against  the  3D  TDH  capabil¬ 
ity  for  mission  planning  and  situational 
awareness  in  various  mission  taskings. 
Mean  results  across  all  tasks  consis¬ 
tently  favored  TDHs  over  conventional 
2D  imagery.  The  data  demonstrated  that 
TDHs  can  improve  accuracy  and  reduce 
the  time  required  to  visually  identify  tacti¬ 
cal  and  terrain-related  fea¬ 
tures,  compared  to  using 
conventional  2D  maps  and 
imagery.  More  recently, 
RDECOM-STTC  and  711 
HPW/RHA  collaborated 
to  evaluate  the  effective¬ 
ness  of  TDHs  used  during 
mission  training  exercises 
conducted  at  the  Joint 
Multinational  Readiness 
Center  at  Hohenfels,  Ger¬ 
many. 

A  visual  scene  of  a  3D 
world  is  a  more  intuitive 
and  natural  representa¬ 
tion  than  a  2D  display, 
and  a  single  integrated 
object  reduces  the  need  for 
mental  integration  of  two 
or  three  separate  represen- 
Scale  Overlay.  tations  (Wickens,  1990;  & 
Rossi,  etal.,  2007).  Joint 
711  HPW/RHA  and  RDECOM-STTC 
research  to  date  indicates  that  TDH  imag¬ 
ery  of  an  area,  when  compared  to  con¬ 
ventional  2D  imagery  of  equal  resolution 
and  scale,  enables  people  to  more  rapidly 
and/or  accurately  identify  subtle  terrain 
profiles,  features,  potential  threats,  and 
obstacles.  711  HPW/RHA  is  conducting 
further  research  to  expand  and  capitalize 
on  the  capabilities  offered  by 
TDHs  in  other  mission  areas 
and  domains.  ^ 


John  Martin,  711  HPW/RHAE 
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There  is  significant  interest  in  the 
exploitation  of  online  interactive  gaming 
environments  for  training  across  a  number 
of  applied  domains  including  military 
operations.  A  number  of  multinational 
research  organizations  are  interested  in 
developing  lower  cost,  high  fidelity  envi¬ 
ronments  based  on  games  that  can  be 
interfaced  with  other  live,  virtual,  and 
constructive  (LVC)  environments  for 
learning.  The  711th  Human  Performance 
Wing,  Human  Effectiveness  Directorate, 
Warfighter  Readiness  Research  Division 
(711  HPW/RHA)  is  conducting  research 
to  explore  game-based  applications  for 
training. 

One  goal  is  to  develop  and  demonstrate 
a  “Family  of  Complementary  Trainers” 
where  game-based  environments  and 
training  are  part  of  an  integrated  train¬ 
ing  “enterprise”  that  includes  games, 
part-task  trainers,  deployable  Simula¬ 


Gaming  and  Training:  Examining  the 
Learning  Potential  of  Games  in  Live, 
virtual,  and  Constructive  Operations 


tors,  full-mission  trainers,  and 
live  operational  systems.  By 
sharing  common  technologies, 
authoring  and  scenario  scripts, 
metrics,  and  tools  for  feedback, 
the  costs  for  each  system  and  for 
the  entire  enterprise  can  be  sub¬ 
stantially  reduced  and  each  system 
can  be  tailored  to  support  training 
and  learning  at  appropriate  levels  of 
fidelity. 

A  second  goal  is  research  activity 
with  a  number  of  universities  around 
the  country  to  establish  first-order 
principles  for  designing  and  delivering 
learning-enabled,  game-based  scenarios. 

Key  research  questions  include:  What 
are  the  specific  and  common  design  fea¬ 
tures  that  are  critical  to  training  systems? 
What  are  the  specific  and  common  design 
features  of  gaming  environments?  Which 
features  of  gaming  environments  facilitate 
or  impede  their  application  as  training  sys¬ 
tems  and  learning  environments?  What 
are  the  essential  changes  to  a  candidate 
gaming  environment  that  must  be  made 
to  apply  it  as  a  training/leaming  environ¬ 
ment?  What  design  recommendations 
and  standards  can  be  developed  for  future 
training  system  and  gaming  environment 
development? 


ment  of  synthetic  task  environments  based 
on  gaming  for  military  and  academic 
experimentation.  The  7 1 1  HP W/RHA  has 
developed  a  prototype  gaming  laboratory 
to  explore  these  research  questions  and  to 
enable  the  Mesa  Research  Site  to  lever¬ 
age  game-based  approaches  to  learning  to 
promote  readiness.  Known  as  the  Game 
Research  Integration  for  Learning  (GRIL) 
laboratory,  the  newest  testbed  includes 
technology  and  tools  for  a  variety  of  com¬ 
mercial,  academic,  and  government  col¬ 
laborators  to  integrate  and  evaluate  with 
the  research  questions  and  goals. 

GRIL  is  leveraging  several  Creative 
Research  and  Development  Agreements 
(CRADA)  to  increase  both  technology 
and  human  resources.  Two  of  these  col¬ 
laborators  will  be  highlighted  at  I/ITSEC 
2009:  Products  from  Aptima  displaying 
a  Distributed  Dynamic  Decision  making 
task  environment,  and  L-3  Communica¬ 
tions  (Bluebox)  will  be  working  together 
through  middleware  developed  in  the 
GRIL.  By  collaborating  with  both  univer¬ 
sities  and  industry  711HPW/RHA  hopes 
to  accelerate  the  process  of 
developing  training  options  for 
the  warfighter.  ^  f  j 


The  711  HPW/RHA  is  also  examining 
the  development  of  an  authoring  system 
for  adding  instructional  components  to 
games  statistics  as  well  as  the  develop¬ 


Lt  Joel  Walker,  711  HPW/RHAS 
Lt  Matthew  Lisa,  711  HPW/RHAE 
Dr.  Winston  Bennett,  Jr.,  711  HPW/RHAS 


711  hpw/rha  Begins  Analysis  of  Joint  Terminal 
Attack  controller  Training  Effectiveness 


In  preparation  for  our  Joint  Terminal 
Attack  Controller  (JTAC)  Warfighter 
Training  Effectiveness  Research  project, 
the  711th  Human  Performance  Wing, 
Human  Effectiveness  Directorate’s  Joint 
Close  Air  Support  (JCAS)  team  began 
conducting  fidelity  studies  in  the  JTAC 
domes  at  the  Mesa  Research  Site.  The 
goal  of  the  fidelity  study  is  to  define  the 
requirements  JTACs  will  need  to  maxi¬ 
mize  the  realism  of  the  training  received 
in  simulated  training  events. 

During  the  months  of  August  and 
September,  members  of  the  13th  Air 
Operations  Squadron  (ASOS)  and  20th 
ASOS  participated  in  Warfighter  Train¬ 
ing  Research  Events  (WTRE).  Two-man 
JTAC  teams  trained  on  several  different 
scenarios  that  incorporated  their  JTAC 


20th  Air  Support  Operations  Squadron  evaluates 
JTAC  performance  during  September  Warfighter 
Training  Exercise.  Photo  by  Bruce  Liddil. 


specific  toolkit  items.  At  the  conclu¬ 
sion  of  WTRE,  the  teams  were  given  an 
opportunity  to  provide  feedback  to  the 


researchers  on  their  experiences.  Spe¬ 
cific  questions  were  asked  to  determine 
their  opinion  on  the  appropriate  level  of 
fidelity,  the  best  mix  of  live  vs.  simulated 
controls  for  maintaining  proficiency,  and 
the  best  dome  configuration  (1/2  vs.  full 
dome)  for  training. 


The  JCAS  team  incorporated  several 
changes  based  on  the  feedback  received 
and  while  conclusive  data  has  not  yet  been 
received,  the  initial  feedback  is  positive. 
The  fidelity  study  will  continue  through 
the  end  of  the  year,  at  which  time  the  team 
will  shift  the  programs  focus  to  JTAC 
training  effectiveness  research 
and  training  gap  analysis. 


Lt  John  Neterer,  711  HPW/RHAE 


Experimental  Deployable  Tactical  Training  Testbed  (X-DTT)  Coes 
to  Aviano  Air  Base,  Italy  for  18-Month  Field  Evaluation 


Live,  Virtual,  and  Constructive  Control  System  planned  for  deployment  to  Aviano 
AB,  Italy  by  summer  2010.  Photo  by  Bruce  Liddil 


The  711  HPW/RHA  researchers  and 
engineers  recently  completed  the  devel¬ 
opment  and  testbed  evaluation  of  a  less- 
than-full-physical-fidelity  simulation 
environment  for  F- 16  air-to-air  and  air-to- 
ground  tactical  training.  They  also  com¬ 
pleted  the  development  of  several  new 
training  support,  management,  and  evalu¬ 
ation  tools  for  debriefing  and  for  routinely 
tracking  and  assessing  performance  and 
proficiency.  All  of  these  innovations  are 
being  deployed  in  an  integrated  package 
to  Aviano  AB,  Italy  for  a  collaborative  Air 
Force  Research  Laboratory,  Air  Combat 
Command,  and  United  States  Air  Forces 
in  Europe  field  evaluation.  Research  and 
operational  readiness  data  will  be  gath¬ 
ered  to  assess  the  impact  and  practical 
utility  of  such  devices  for  routine  training. 

The  fielded  environment  provides 
researchers,  engineers,  and  operators 
with  a  capability  to  systematically  assess 
the  “tradespace”  for  environment  fidelity 
while,  at  the  same  time,  providing  a  robust 
training  capability  for  operational  person¬ 
nel.  In  other  words,  it  will  be  possible  to 
objectively  determine  “how  much  of  what 
kind  of  fidelity  is  needed  to  train  a  given 
set  of  tactical  tasks?” 

We  have  recently  developed,  validated, 
and  demonstrated  a  subjective-rating 
and  objective  mission  performance  data- 
driven  integrated  methodology  for  evalu¬ 
ating  simulation  environment  fidelity. 
The  methodology  permits  quantifying 
the  impact  of  different  levels  of  fidel¬ 
ity  on  training  effectiveness  and  transfer. 
The  process  can  be  used  to  support  a  new 
generation  of  simulation  environment 
assessment  and  certification  processes 
where  fidelity  baselines  are  established 


and  compared  to 
expected  values 
to  identify  gaps. 

Technical 
activities  to  sup¬ 
port  the  study 
and  the  deploy¬ 
able  trainer  con¬ 
cepts  leveraged 
the  711  HPW/ 
RHA  pioneering 
work  in  the  appli¬ 
cation  of  versa 
module  europa- 
bus  architecture- 
based  simulation 
environments 
and  extensive 
training  and 
instructional  design  research.  Research¬ 
ers  and  engineers  defined  and  developed 
a  deployable  and  reconfigurable  tactical 
training  environment  and  tools  that  sup¬ 
port  training  and  after  action  review  along 
with  routine  performance  measurement 
and  tracking.  The  711  HPW/RHA  has 
also  demonstrated  and  validated  a  capa¬ 
bility  to  rapidly  load  different  operational 
flight  program  (OFP)  software,  tools, 
databases  and  models  in  the  deployable 
trainers  while  maintaining  a  reasonable 
level  of  tactical  and  instructional  fidelity 
and  validity.  This  capability  to  load,  run, 
train,  download,  load  new,  run,  and  train 
multiple  OFPs  and  related  targeting  pod 
models  and  simulations  in  the  same  envi¬ 
ronment  in  a  very  short  turn  time  is  crucial 
to  this  community  as  there  are  OFPs  in  the 
F-16  fleet  today  that  represent  different 
capabilities  and  mission  parameters. 

The  data  from  this  field 
evaluation  will 
inform  several 
ongoing  areas  of 
research  includ¬ 
ing:  (a)  the  applica¬ 
tion  of  performance 
measurement  and 
visualization  tools  to 
after  action  review;  (b)  the 
development  of  a  more  com¬ 
prehensive  approach  to  mixing 
live  and  virtual  training  oppor¬ 
tunities;  (c)  evidence-based 
approach  to  simulation  certifica¬ 
tion  and  accreditation  for  training; 

(d)  the  level  of  fidelity  required  to  train 
specific  training  tasks;  and  (e)  assessing 
the  feasibility  of  tools  for  routinely  track¬ 
ing  performance  and  proficiency  to  better 


inform  squadron. 

Finally,  as  part  of  an  ongoing  collabora¬ 
tion  with  the  Naval  Air  Systems  Command 
in  Orlando  and  the  Navy’s  PMA-205,  we 
have  invited  our  colleagues  to  participate 
with  us  in  the  evaluations  and  to  use  the 
Aviano  deployment  as  an  opportunity  to 
gather  data  for  deployable  trainers  for 
Navy  and  Marine  Corps  operators.  Les¬ 
sons  learned  from  our  development  work 
with  the  F-16  community  is  informing  our 
design  work  for  a  reconfigurable  F-16/F- 
35  deployable  training  concepts  and  envi¬ 
ronment.  The  first  two-ship  will  be  in  the 
field  at  Aviano  by  mid-December.  A  full 
four-ship  is  expected  to  be  in  place  by 
summer  2010. 

Feedback  data  from  comparative  fidel¬ 
ity  studies  being  conducted  with  the 
Experimental  Deployable  Tactical  Trainer 
(X-DTT)  and  our  full  fidelity  simulations 
has  indicated  that  a  more  full  field  of 
regard  for  the  X-DTTs  would  be  worthy 
of  exploration.  Therefore,  we  are  looking 
at  alternative  approaches  to  increase  the 
field  of  view  for  the  X-DTTs  to  provide 
a  more  realistic  environment  for  training. 
The  alternatives  must  be  easily  sustain¬ 
able,  requiring  limited  maintenance  and 
alignment  so  that  field  support  and  routine 
adjustments  are  not  required  to  maintain 
good  training  capability.  The  711  HPW/ 
RHA  expects  to  showcase  one  or  more 
alternatives  very  soon  as  part 
of  the  continuing  program 
of  research. 


Dr.  Winston  Bennett,  Jr.,  711  HPW/RHAS 


Experimental 
Deployable 
Tactical  Trainer 


TARGETS  DF  OPPORTUNITY 
Game-Based  Defensive  Counter  space  Training  Exemplar  Transitions  to  Field 


The  711th  Human  Performance  Wing, 
Human  Effectiveness  Directorate,  War¬ 
fighter  Readiness  Research  Division 
(711  HPW/RHA),  in  collaboration  with 
Sonalysts,  Inc.  Aptima,  Inc.,  and  the 
USAF  Space  education,  training,  and 
operational  communities,  recently  com¬ 
pleted  the  development  and  transition  of 
a  new  gaming  and  training  environment 
for  defensive  counter  space  operations. 
This  effort  directly  aligns  with  ongoing 
711  HPW/RHA  research  evaluating  how 
games  can  best  be  used  to  support  opera¬ 
tional  training,  rehearsal,  and  exercise.  It 
also  responded  to  a  specific  need  for  effec¬ 
tive  training  in  counter  space  operation 
and  for  tools  to  enable  the  USAF  Space 
community  to  examine  alternative  tactics, 
techniques,  and  procedures  that  will  con¬ 
tribute  to  their  ability  to  detect,  identify, 
track,  and  locate  or  disrupt  national  space 
asset  threats  in  a  flexible  and  realistic 
environment.  As  the  costs  associated  with 


high  fidelity  modeling  and  simulation- 
based  training  continue  to  skyrocket,  there 
is  a  substantial  opportunity  to  evaluate 
games  and  game-based  tools  and  fidelity 
for  addressing  cost  and  training  gaps  that 
exist  in  the  operational  Air  Force  today. 
Moreover,  the  USAF  Space  community  in 
particular  and  the  US  space  community  in 
general  do  not  have  a  capability  to  exam¬ 
ine  alternative  approaches  for  those  types 
of  space  operations  in  a  realistic  envi¬ 
ronment.  The  711  HPW/RHA  developed 
a  gaming  environment  where  specific 
space  operations  tasks  can  be  trained  and 
rehearsed  in  a  realistic  set  of  scenarios  and 
simulations.  The  environment  includes 
object  models  to  simulate  interactions 
between  satellites  and  ground  stations 
plus  a  capacity  for  multiple  players  to  par¬ 
ticipate,  providing  command  and  control, 
tactical,  and  operational  information.  The 
team’s  solution  applies  gaming  and  train¬ 
ing  system  development  expertise  with 


advanced  human  engineering  and  perfor¬ 
mance  expertise  to  demonstrate  an  inno¬ 
vative  and  economical  training  system. 
Leveraging  both  commercial  gaming 
engines  and  tools,  along  with  scenario 
design  and  embedded  robust  individual 
and  team  performance  measurement,  the 
team  developed  and  evaluated  conceptu¬ 
ally  valid  scenarios  of  sufficient  fidelity  to 
support  the  key  operational  tasks.  It  also 
allowed  team  researchers  to  examine  and 
evaluate  alternative  approaches  to  instruc¬ 
tion  and  assessment  within  a  robust  and 
commercially  viable  gaming  architecture. 
This  project  is  transitioning  to  the  space 
community  as  a  Phase  III  Small  Business 
Innovation  Research  effort  where  addi¬ 
tional  development  work  will  continue 
and  where  a  series  of  controlled 
field  evaluations  are  planned. 

Dr.  Winston  Bennett,  Jr.,  711 HPW/RHAS 


BRIEFS  AND  DEBRIEFS 


711  hpw/rha  Exercise  Recognized  by  community  of  interest 


The  711th  Human  Performance  Wing, 
Human  Effectiveness  Directorate,  War¬ 
fighter  Readiness  Research  Division  (711 
HPW/RHA)  joint  Training  Research  Exer¬ 
cise  (T-REX)  has  been  approved  by  Air 
Combat  Command  as  a  minor  exercise  for 
Air  and  Space  Operation  Center  (AOC) 
training  requirements.  T-REX  research 
trials  use  team  level  instructional  design 
approaches,  creating  a  performance  mea¬ 
surement  development  research  venue 
for  team  decision  making  and  kill  chain 
execution. 

The  trials,  while  having  high  research 
validity,  are  also  designed  to  provide  a 
realistic  and  immersive  environment  for 
integration  and  testing  of  emerging  AOC 
technologies  and  methodologies.  T-REX 
trials  are  conducted  twice  a  year.  Minor 
exercise  status  allows  RHA’s  AOC  Train¬ 
ing  Research  team  to  deconflict  with  other 
major  and  minor  exercises  and  lessens 
the  burden  on  the  experienced  AOC  War¬ 
fighters  that  are  vital  to  data  validation 
of  T-REX.  This  allows  data  pooling  and 
trend  identification. 


Recognizing  T-REX  as  a  minor  exer¬ 
cise  is  not  only  a  major  win  for  the 
research  team,  but  for  the  operational 
community  as  well.  It  expands  the  pool 
of  research  subjects  available,  but  also 
provides  increased  selectivity  of  appro¬ 
priate  subjects  for  the  training  events 
and  research  protocols.  Although  the 
focus  of  the  event  is  research,  the  par¬ 
ticipants  receive  excellent  training  on 
emerging  techniques  at  the  operational 
level.  Participating  in  our  exercise  now 
fulfills  the  Warfighters’  annual  require¬ 
ment  to  train  at  one  major  and  three 
minor  exercises. 

The  more  qualified  participants  possess 
the  expertise  required  to  provide  feedback 
and  performance  data  to  be  analyzed  to 
answer  research  questions  and  to  further 
AOC  training  research  while  increasing 
our  operational  credibility  in  this  impor¬ 
tant  community.  T-REX  is  reaching  the 
same  level  of  effectiveness  as  our  F-16 
research.  Like  the  Viper  community,  the 
AOC  cadre  is  recognizing  the  importance 
of  this  exercise  and  are  pushing  them- 


Observers  measure  warfighter  performance  during  train¬ 
ing  research  exercise.  Photo  by  Bruce  Liddil. 


selves  and  other  available  command  and 
control  experts  to  return  to  the  7 1 1  HPW/ 
RHA  for  the  next  event. 

Finally,  this  accreditation  will  strengthen 
the  AOC  training  research  and  operational 
stakeholder  relationship  ensuring  that 
research  products  are  transitioned  more 
rapidly  and  completely  to  the  com-  ^ 
munities  of  interest  and  practice. 


2Lt  Cameron  Barnes,  711  HPW/RHAS 
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